Image processing device, image processing method, and image system

ABSTRACT

There is provided an image processing device, an image processing method, and an image system capable of reducing discomfort of a user, such as “VR sickness.” The image processing device includes: an image generation unit configured to generate an avatar viewpoint image according to a viewpoint of an avatar corresponding to a user in a virtual world as a first presentation image to be presented to the user, and generate a second presentation image different from the avatar viewpoint image in a case in which posture difference occurs between an actual posture of the user based on a result of detecting motion of the user and a posture of the avatar. The present technique is applicable, for example, to an image processing device etc. for generating an image to be displayed on a head mounted display.

CROSS REFERENCE TO PRIOR APPLICATION

This application is a National Stage Patent Application of PCTInternational Patent Application No. PCT/JP2018/004567 (filed on Feb. 9,2018) under 35 U.S.C. § 371, which claims priority to Japanese PatentApplication No. 2017-032021 (filed on Feb. 23, 2017), which are allhereby incorporated by reference in their entirety.

TECHNICAL FIELD

The present technique relates to an image processing device, an imageprocessing method, and an image system. In particular, the presenttechnique relates to an image processing device, an image processingmethod, and an image system capable of reducing discomfort of a user,such as “VR sickness.”

BACKGROUND ART

In recent years, head-mounted displays and head-up displays have becomecapable of presenting images with a wide viewing angle. Bychanging/following (tracking) a direction and a posture of the image tobe displayed naturally in response to the posture of the body or head,more realistic first-person viewpoint experiences are becoming feasible.The first-person viewpoint, which is particularly called first-personshooting (F.P.S.) in the video game world, is becoming popular.

On the other hand, it is known that some users may experience discomfortcalled “VR sickness” or “video sickness” when experiencing virtualreality (VR) using a head mounted display or a head up display. Thisdiscomfort is said to occur when there is a gap between the sense thatthe user feels or the user predicts from the experience and the actualsense (e.g., visual stimulation).

Contrivance for reducing the discomfort of this “VR sickness” or “videosickness” has been proposed (see, for example, PTL 1).

CITATION LIST Patent Literature

[PTL 1]

JP 2016-31439A.

SUMMARY Technical Problem

However, measures against such discomfort of “VR sickness” etc. arefurther required.

The present technique has been made in view of such a situation, and isintended to be able to reduce the discomfort of the user such as “VRsickness.”

Solution to Problem

An image processing device according to one aspect of the presenttechnique includes animate generation unit configured to generate anavatar viewpoint image according to a viewpoint of an avatarcorresponding to a user in a virtual world as a first presentation imageto be presented to the user, and generate a second presentation imagedifferent from the avatar viewpoint image in a case in which a posturedifference occurs between an actual posture of the user based on aresult of detecting motion of the user and a posture of the avatar.

An image processing method according to one aspect of the presenttechnique includes the step of: by an image processing device,generating an avatar viewpoint image according to a viewpoint of anavatar corresponding to a user in a virtual world as a firstpresentation image to be presented to the user and generating a secondpresentation image different from the avatar viewpoint image in a casein which a posture difference occurs between an actual posture of theuser based on a result of detecting motion of the user and a posture ofthe avatar.

An image system according to one aspect of the present techniqueincludes an image generation unit configured to generate an avatarviewpoint image according to a viewpoint of an avatar corresponding to auser in a virtual world as a first presentation image to be presented tothe user, and generate a second presentation image different from theavatar viewpoint image in a case in which a posture difference occursbetween an actual posture of the user based on a result of detectingmotion of the user and a posture of the avatar; and a display unitconfigured to display the first presentation image and the secondpresentation image.

In one aspect of the present technique, an avatar viewpoint image isgenerated according to a viewpoint of an avatar corresponding to a userin a virtual world as a first presentation image to be presented to theuser, and a second presentation image different from the avatarviewpoint image is generated in a case in which a posture differenceoccurs between the actual posture of the user based on a result ofdetecting motion of the user and a posture of the avatar.

Note that the image processing device according to one aspect of thepresent technique may be achieved by causing a computer to execute aprogram.

In addition, in order to achieve the image processing device accordingto one aspect of the present technique, the program to be executed bythe computer may be provided by transmitting via a transmission mediumor recording on a recording medium.

The image processing device may be an independent device or an internalblock constituting one device.

Advantageous Effect of Invention

According to one aspect of the present technique, the discomfort of theuser such as “VR sickness” may be reduced.

In addition, the effect described here is not necessarily limited, andmay be any effect described in the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 illustrates an exemplary configuration of an embodiment of animage system to which the present technique is applied.

FIG. 2 illustrates an example of a content image of a boxing game.

FIG. 3 illustrates a block diagram representing a detailed exemplaryconfiguration of the imaging system of FIG. 1.

FIG. 4 illustrates a figure explaining a coordinate system of a virtualworld.

FIG. 5 illustrates a figure explaining three postures including a userposture, an avatar posture, and a presentation image posture.

FIG. 6 illustrates a flowchart explaining a presentation imagegeneration process.

FIG. 7 illustrates a figure explaining an example of contents in which aplurality of users shares a virtual world.

FIG. 8 illustrates a block diagram representing an exemplaryconfiguration of an embodiment of a computer to which the presenttechnique is applied.

DESCRIPTION OF EMBODIMENTS

Hereinafter, a mode for carrying out the present technique (hereinafter,referred to as an embodiment) will be described.

<Exemplary Configuration of an Image System>

FIG. 1 illustrates an exemplary configuration of an embodiment of animage system to which the present technique is applied.

The image system 1 of FIG. 1 includes a head mounted display 11(hereinafter, referred to as an HMD 11), an image processing device 12,and a controller 13.

The HMD 11 displays an image in the virtual world generated by the imageprocessing device 12 to present it to a user. The image processingdevice 12 generates an image to be displayed on the HMD 11 to provide itto the HMD 11. The controller 13 includes a plurality of handlingbuttons (not illustrated), receiving handling of the user to provide theimage processing device 12 with a predetermined instruction in responseto the handling of the user.

A wired cable such as HDMI (High Definition Multimedia Interface) or MHL(Mobile High-definition Link), or wireless communication such as Wi-Fi(Wireless Fidelity), wireless HD or Miracast, for example, connectsbetween the HMD 11 and the image processing device 12, and between theimage processing device 12 and the controller 13.

In the present embodiment, for example, the image processing device 12causes an image as illustrated in FIG. 2 to be displayed. The image inFIG. 2 illustrates an image of a first-person viewpoint boxing game(content) in which the user wearing the HMD 11 is a player. The imagedisplayed on the HMD 11 may be a 2D image. This image may also be a 3Dimage, which enables three-dimensional visual recognition by displayinga right-eye image for presenting to the user's right eye and a left-eyeimage for presenting to the user's left eye.

In the image system 1 configured as described above, the imageprocessing device 12 generates an image and cause the HMD 11 to displaythe image so as to reduce discomfort such as “VR sickness” in a state inwhich the user visually recognizes the image displayed on the HMD 11 andexperiences the virtual world.

In addition, the HMD 11, the image processing device 12, and thecontroller 13 may be combined as needed, and the image system 1 may beconfigured by one or two devices.

<Detailed Exemplary Configuration of an Image System>

FIG. 3 illustrates a block diagram representing a detailed exemplaryconfiguration of the imaging system 1.

The HMD 11 includes a sensor 41 and a display unit 42.

The sensor 41 includes, for example, a combination of one or more sensorelements such as a gyro sensor, an acceleration sensor, and ageomagnetic sensor, to detect the position and direction of the head ofthe user. In the present embodiment, the sensor 41 is assumed to includea sensor capable of detecting a total of nine axes of a three-axis gyrosensor, a three-axis acceleration sensor, and a three-axis geomagneticsensor. The sensor 41 provides the detected result to the imageprocessing device 12.

The display unit 42 includes, for example, an organic EL(Electro-Luminescence) element, a liquid crystal display, etc., anddisplays a predetermined image on the basis of an image signal providedfrom the image processing device 12.

In addition, the HMD 11 may have a speaker that outputs sound, amicrophone that acquires the sound of the user, etc.

The controller 13 includes a handling unit 81 including a plurality ofhandling buttons, receiving the handling of the user to provide theimage processing device 12 with a predetermined instruction in responseto the handling of the user.

The image processing device 12 includes at least a storage unit 61 andan image generation unit 62. The image generation unit 62 includes anavatar operation control unit 71, a user posture detection unit 72, aposture difference calculation unit 73, and a presentation imagegeneration unit 74.

The storage unit 61 includes, for example, a hard disk, a non-volatilememory, etc., and stores programs for controlling the operation of theimage processing device 12 and contents (programs) for displaying animage on the HMD 11, etc.

The avatar operation control unit 71 controls the operation (posture) ofan avatar corresponding to the user in the image (content image) in thevirtual world displayed on the display unit 42. The control of theoperation. (posture) of the avatar also takes into consideration thereal posture of the user (the actual posture of the user) detected bythe user posture detection unit 72. The operation of the avatar changes,for example, in the content image of the virtual world displayed on thedisplay unit 42, depending on the operation selected by the user, theaction performed by the user, etc.

As an opponent in the boxing game illustrated in FIG. 2, the avataroperation control unit 71 also controls, for example, the operation(posture) of an avatar of another user in a case in which an avatarother than the player appears in the image displayed on the display unit42. In the following, in order to distinguish the avatar of the userfrom the avatar of another user, in a case in which the avatar ofanother user is intended, it is referred to as another avatar, and inthe case of simply referring to as an avatar, at means the user's ownavatar.

The user posture detection unit 72 detects the actual posture of theuser on the basis of the detection result of the position and directionof the head of the user provided from the sensor 41.

The posture difference calculation unit 73 calculates the a posturedifference between the posture of the avatar determined by the avataroperation control unit 71 and the actual posture of the user detected bythe user posture detection unit 72.

The presentation image generation unit 74 generates a presentation imageto be presented to the user, which is an image of the content acquiredfrom the storage unit 61, and provides the presentation image to thedisplay unit 42. The presentation image is generated on the basis of theoperation of the user's own avatar and another avatar determined by theavatar operation control unit 71, the actual posture of the userdetermined by the user posture detection unit 72, and the posturedifference determined by the posture difference calculation unit 73between the posture of the avatar and the actual posture of the user. Ina normal scene, the presentation image generation unit 74 aligns theposture of the avatar with the actual posture of the user, and generatesa presentation image (first presentation image) such that an imageaccording to the viewpoint of the avatar is presented to the user. Undera predetermined condition, in order to reduce the discomfort of the usersuch as “VR sickness,” as image different from the image of the avatarviewpoint is generated as a presentation image (second presentationimage) and causes the display unit 42 to display the image.

Description of the Presentation Image>

The presentation image generated by the presentation image generationunit 74 will be further described with reference to FIGS. 4 and 5.

First, FIG. 4 describes the virtual world in the content displayed onthe display unit 42 and a coordinate system that expresses the postureof the avatar on the virtual world.

Assuming that the virtual world in the content displayed on the displayunit 42 is a three-dimensional space including the X axis, Y axis, and Zaxis as illustrated in A of FIG. 4, the posture of the avatar may beexpressed by five parameters (x, y, z, θ, φ) which includes athree-dimensional position (x, y, z) and a two-dimensional direction.(θ, φ).

Here, the three-dimensional position (x, y, z) corresponds to theposition of the head of the avatar on the virtual world, and thetwo-dimensional direction (θ, φ) corresponds to a gaze direction. (headdirection) of the avatar on the virtual world. As illustrated in B ofFIG. 4, the direction θ is a so-called azimuth angle, which is formed bythe gaze direction with respect to a predetermined reference axis (Zaxis in the example of FIG. 4B) on the XZ plane. As illustrated in C ofFIG. 4 the direction φ is a so-called elevation angle, which is an anglein the Y-axis direction formed by the gaze direction with respect to theXZ lane. Therefore, the posture in the present specification includesnot only the head position on the virtual world but also the gazedirection.

In the virtual world of the content displayed on the display unit 42,three postures of the user posture pos_(u), the avatar posture pos_(a),and the presentation image posture pos_(v) are defined as follows.user posture pos_(u)=(x _(u) ,y _(u) ,z _(u),θ_(u),φ_(u))avatar posture pos_(a)=(x _(a) ,y _(a) ,z _(a),θ_(a),φ_(a))presentation image posture pos_(v)=(x _(v) ,y _(v) ,z _(v),θ_(v),φ_(v))

The user posture pos_(u) is a projection of the actual posture of theuser determined by the detection result provided from the sensor 41 onthe virtual world. The avatar posture pos_(a) is a posture of the avatarexpressed by coordinates on the virtual world. The presentation imageposture pos_(v) is a virtual posture corresponding to the presentationimage displayed on the display unit 42.

FIG. 5 specifically describes the three postures of the user posturepos_(u), the avatar posture pos_(a), and the presentation image posturepos_(v).

These three postures are described using an example of the content ofthe boxing game illustrated in FIG. 2.

For example, since the posture of the avatar is controlled according tothe actual posture of the user detected by the user posture detectionunit 72 in a normal scene, the avatar posture pos_(a) and the userposture pos_(u) are the same. That is, avatar posture pos_(a)=userposture pos_(u). Further, since the presentation image displayed on thedisplay unit 42 is an image of the virtual world from the viewpoint ofthe avatar, presentation image posture pos_(v)=avatar posture pos_(a).

Therefore, in the normal scene where the special process to reduce “VRsickness” is not performed, presentation image posture pos_(v)=avatarposture pos_(a)=user posture pos_(u).

Then, an example is described assuming the scene where the avatar is onthe canvas (falling to the ground) by the opponent's punch.

Assuming that the actual user does not fall to the ground and stands upas in a posture 101 of FIG. 5, the user posture pos_(u) calculated bythe user posture detection unit 72=(x_(u), y_(u), z_(u), θ_(u), φ_(a))has the posture 101.

By contrast, the avatar posture pos_(a)=(x_(a), y_(a), z_(a), θ_(a),φ_(a)) controlled by the avatar operation control unit 71 has a posture102 in which the avatar falls to the ground as illustrated in FIG. 5.

In a normal scene, as described above, the image of the virtual worldfrom the viewpoint of the avatar is used as the presentation image.However, if the presentation image is greatly changed in accordance withrapid movement of the avatar, the user may cause possible “VR sickness.”

Therefore, the presentation image generation unit 74 generates not theimage corresponding to the avatar posture pos_(a) (the posture 102 inFIG. 5) but, for example, an image 111 corresponding to a posture 103close to the actual posture of the user (the posture 101 in FIG. 5) as apresentation image, and causes the display unit 42 to display the image111. The image 111 as the presentation image is an image whose viewchange is more gradual than the image corresponding to the avatarposture pos_(a). The posture 103 corresponding to the image 111 to bethe presentation image is the presentation image posture pos_(v)=x_(v),y_(v), z_(v), θ_(v), φ_(v)).

In the scene of the example of FIG. 5 subjected to the process forreducing VR sickness, the presentation image posture posy is differentfrom any posture of the avatar posture pos_(a) and the user posturepos_(u), but it is sufficient that the presentation image is an imagedifferent from an avatar viewpoint image that is at least thepresentation image in the normal scene. For example, the presentationimage may be an image corresponding to the actual posture of the user.

<Presentation Image Generation Process>

Then, with reference to a flowchart of FIG. 6, described is apresentation image generation process by the image processing device 12.This process is started, for example, when handling is performed tocause the display unit 42 of the HMD 11 to display a predeterminedcontent image.

First, in step S1, the avatar operation control unit 71 determines theoperation of the avatar (avatar posture pos_(a)) in the virtual worlddisplayed on the display unit 42. In a case in which another avatarother than the player himself appears in the virtual world, the avataroperation control unit 71 also determines the operation (posture) ofanother avatar. After step S1, determination of the operation of theavatar is always executed according to the content.

In step S2, the user posture detection unit 72 detects the actualposture pos_(u) of the user on the basis of the sensor detection resultindicating the position and direction of the head of the user providedfrom the sensor 41. Thereafter, the detection of the actual posturepos_(u) of the user based on the sensor detection result is alsoconstantly executed. The detected actual posture pos_(u) of the user isprovided to the avatar operation control unit 71, and the operation(posture) of the avatar is matched to the operation of the user.

In step S3, the presentation image generation unit 74 generates anavatar viewpoint image that is an image from the viewpoint of theavatar, as a presentation image, providing the generated avatarviewpoint image to the display unit 42 of the HMD 11 to cause thedisplay unit 42 to display the supplied image. The presentation imageposture pos_(v) here is equal to the avatar posture pos_(a) and the userposture pos_(u) (presentation image posture pos_(v)=avatar posturepos_(a)=user posture pos_(u)).

In step S4, the posture difference calculation unit 73 calculates aposture difference between the avatar posture pos_(a) determined by theavatar operation control unit 71 and the actual posture pos_(u) of theuser detected by the user posture detection unit 72, and provide thecalculated posture difference to the presentation image generation unit74.

In step S5, the presentation image generation unit 74 determines whetherthe posture difference between the avatar posture pos_(a) and the actualposture pos_(u) of the user is equal to or greater than a predeterminedfirst range.

Here, when, as a first range in step S5, the threshold thres_(θ) of theazimuth angle θ and the threshold thres_(φ) of the elevation angle φ areset, and either condition of the following equation (1) or equation (2)is satisfied, the presentation image generation unit 74 may determinethat the posture difference between the avatar posture pos_(a) and theactual posture pos_(u) of the user is equal to or greater than the firstrange. In the equation (1) and the equation (2), |d| represents theabsolute value of d.|θ_(a)−θ_(u)|>thres_(θ)  (1)|φ_(a)−φ_(u)|>thres_(φ)  (2)

In the equation (1), thres_(θ) is a threshold of the azimuth angle θ,and thres_(φ) in the equation (2) is a threshold of the elevation angleφ. The threshold thres_(θ) and the threshold thres_(φ) may be set to anyvalue, and for example, may be set to 30 degrees. This numerical valueof 30 degrees is a value where the average human moves the gazedirection by moving only the eyeball when looking at an object that isoff the front within 30 degrees, while the average human performs headmotion naturally when looking at an object in a direction that isfurther out of 30 degrees.

In addition, in the case of using severe determination conditions forpreventing sickness, the threshold thres_(θ) and the threshold thres_(φ)may be set to one degree. This is because the range of the centralretinal area, called the fovea used to focus on the details of things,is said to be approximately one degree in an average human. Thethreshold thres_(θ) and the threshold thres_(φ) may be changed to thevalues desired by the user using a setting screen, etc. Changing thesethreshold values enables to control the degree for preventing sickness.

The above-described determination condition is mainly the condition thatdetermines that the head direction has been moved. In a case in whichsuch a condition that the avatar may be greatly moved with its entirebody is also taken into consideration, the condition of the followingequation (3) can be added to the conditions of equation (1) and equation(2). That is, when any one of the equations (1) to (3) is satisfied, theposture difference between the avatar posture pos_(a) and the actualposture pos_(u) of the user is determined to be equal to or greater thanthe first range.Δx _(au)>thres_(x)  (3)

In the equation (3), Δx_(au) represents a difference between the headposition (x_(u), y_(u), z_(u)) of the actual posture pos_(u) of the userand the head position (x_(a), y_(a), z_(a)) of the avatar posturepos_(a). The two-dimensional norm of the equation (4) or theone-dimensional norm of the equation (5) may be adopted as Δx_(au). Inthe equation (3), thres_(x) represents the threshold of Δx_(au).[Math. 1]Δx _(au)=√{square root over ((x _(u) −x _(a))²+(y _(u) −y _(a))²+(z _(u)−z _(a))²)}  (4)Δx _(au) =|x _(u) −x _(a) |+|y _(u) −y _(a) |+|z _(u) −z _(a)|  (5)

A moving speed of the head position of the avatar or an angular velocityin the gaze direction may be used as the determination condition in theother step S5. Specifically, assuming that the angular velocity in thegaze direction (θ_(a), φ_(a)) of the avatar is Δθ and Δφ, and a timechange amount (moving velocity) of the head position (x_(a), y_(a),z_(a)) of the avatar is v_(a) the posture difference between the avatarposture pos_(a) and the actual posture pos_(u) of the user may bedetermined to be equal to or greater than the first range when any oneof the equations (6) to (8) is satisfied,|Δθ_(a)−Δθ_(a)>thres_(Δθ)  (6)|Δφ_(a)−Δφ_(a)|>thres_(Δφ)  (7)v _(a)>thres_(v)  (8)

Furthermore, in step S5, it may be configured that, when a phenomenon inwhich the posture difference between the avatar posture pos_(a), and theactual posture pos_(u) of the user is determined to be equal to orgreater than the first range occurs not only one time but also apredetermined number of times or more, or occurs continuously for apredetermined period of time, the posture difference may be alsodetermined to be equal to or greater than the first range.

In a case in which the posture difference between the avatar posturepos_(a) and the actual posture pos_(u) of the user is determined not tobe equal to or greater than the first range in step S5, the processreturns to step S3, and the process of steps S3 to 35 described above isrepeated. Accordingly, in a case in which the posture difference betweenthe avatar posture pos_(a) and the actual posture pos_(u) of the user issmaller than the first range, the avatar viewpoint image from theviewpoint of the avatar is displayed as the presentation image,resulting in presentation image posture pos_(v)=avatar posture pos_(a).

Conversely, in a case in which the posture difference between the avatarposture pos_(a) and the actual posture pos_(u) of the user determined tobe equal to or greater than the first, range in step S5, the processproceeds to step S6. Then the presentation image generation unit 74generates the reduction processing image for reducing discomfort such as“VR sickness” as a presentation image, providing the generated image tothe display unit 42 to cause the display unit 42 to display the suppliedimage.

The presentation image generation unit 74 generates, for example, animage corresponding to the user posture pos_(a) as the reductionprocessing image and causes the display unit 42 to display the generatedimage. In this case, the presentation image posture pos_(v) is the userposture pos_(u).pos_(v)=(x _(v) ,y _(v) ,z _(v),θ_(v),φ_(v))=(x _(u) ,y _(u) ,z_(u),θ_(u),φ_(u))

Alternatively, the presentation image generation unit 74 generates, forexample, an image with the head position of the avatar posture pos_(a)for the head position of the presentation image posture pos_(v), or animage with the gaze direction of the user posture pos_(u) for the gazedirection of the presentation image posture pos_(v), as the reductionprocessing image and causes the display unit 42 to display the generatedimage. That is, the presentation image posture pos_(v) is as follows.pos_(v)=(x _(v) ,y _(v) ,z _(v),θ_(v),φ_(v))=(x _(a) ,y _(a) ,z_(a),θ_(a),φ_(a))

Alternatively, the presentation image generation unit 74 generates, forexample, an image with the head position and the azimuth angle θ of theavatar posture pos_(a) for the head position and the azimuth angle θ ofthe presentation image posture pos_(v), or an image with the elevationangle φ of the user posture pos_(u) for the elevation angle φ of thepresentation image posture pos_(v), as the reduction processing imageand causes the display unit 42 to display the generated image. That is,the presentation image posture pos_(v) is as follows.pos_(v)=(x _(v) ,y _(v) ,z _(v),θ_(v),φ_(v))=(x _(a) ,y _(a) ,z_(a),θ_(a),φ_(u))

This equation means that, since the user is more likely to get sick withthe field of view unintentionally changed in the vertical direction, thegaze direction of the user is adopted only in this direction.

In contrast, for example, as the reduction processing image, an imagewith the head position and the elevation angle φ of the avatar posturepos_(a) for the head position and the elevation angle φ of thepresentation image posture pos_(v), or an image with the azimuth angle θof the user posture pos_(u) for the azimuth angle θ of the presentationimage posture pos_(v), may be generated to be displayed. That is, thepresentation image posture pos_(v) is as follows.pos_(v)=(x _(v) ,y _(v) ,z _(v),θ_(v),φ_(v))=(x _(a) ,y _(a) ,z_(a),θ_(a),φ_(a))

This means that the gaze direction of the user is adopted only for theazimuth angle θ.

Then, in step S7, the presentation image generation unit 74 determineswhether or not the posture difference between the avatar posture pos_(a)and the actual posture pos_(u) of the user is within a predeterminedsecond range.

At step S7, in a case in which the posture difference between the avatarposture pos_(a) and the actual posture pos_(u) of the user is not yetwithin the second range, that is, the posture difference between theavatar posture pos_(a) and the actual posture pos_(a) of the user isdetermined to be greater than the second range, the process returns tostep S6, and the above-described process is repeated. That is, thereduction processing image is continuously generated as the presentationimage and displayed on the display unit 42.

Conversely, in a case in which the posture difference between the avatarposture pos_(a) and the actual posture pos_(u) of the user is determinedto be within the second range in step S7, the process proceeds to stepS8, and the presentation image generation unit 74 generates thereduction processing image so as to gradually approach the avatarviewpoint image (an image corresponding to the avatar posture pos_(a)),then causes the display unit 42 to display the generated image. Afterthe process of step S8, presentation image posture pos_(v)=avatarposture pos_(a)=user posture pos_(u).

For control of the presentation image posture pos_(v) that brings thepresentation image (reduction processing image) continuously close tothe avatar viewpoint image, which is the process of step S8, forexample, calculations according to simple linear motion, curvilinearmotion such as Bezier, or motion within the drive range allowed bygeometric constraints based on a body model may be used.

After step S8, the process returns to step S3, and the above-describedsteps S3 to S8 are executed again. The values of the first range and thesecond range may be the same value or different values.

The above process is executed as a presentation image generationprocess.

As described above, in the presentation image generation process by theimage system 1, is a case in which the posture difference between theavatar posture pos_(a) and the actual posture pos_(u) of the user issmaller than the predetermined threshold (the first range), thepresentation image generation unit 74 generates the avatar viewpointimage according to the viewpoint of the avatar (an image correspondingto the avatar posture pos_(a)) as the first presentation image, andcauses the display unit 42 to display the generated image. In a case inwhich the posture difference between the avatar posture pos_(a) and theactual posture pos_(u) of the user is equal to or greater than apredetermined threshold (first range) as well as the posture differencebetween the avatar posture pos_(a) and the actual posture pos_(u) of theuser occurs above a certain level, the presentation image generationunit 74 generates the reduction processing image different from theavatar viewpoint image as the second presentation image and causes thedisplay unit 42 to display the generated image. This system enablesreduction of the discomfort of the user such as “VR sickness.”

Note that, in the generation of the reduction processing image in stepS6 described above, the reduction processing image is generated anddisplayed using the content image in the virtual world. The imagedifferent from the content image being displayed, for example, a blackimage (blackout screen) in which the entire screen is black only, or analert screen that explains the screen being changed in order to prevent“sickness,” may be displayed.

In this case, in the process executed in step S8 to bring the imagecontinuously close to the avatar viewpoint image, the image may bereturned to the avatar viewpoint image by using an alpha blendingprocess, etc. in which alpha blending of the black image and the avatarviewpoint image is performed.

Assuming that the avatar viewpoint image is I_(a) and the black image isI_(b), the reduction processing image I_(u) in step S8 is expressed bythe following equation (9) using a blending ratio α (0≤α≤1).I _(u) =αI _(a)+(1−α)I _(b)  (9)

By changing the blending ratio α in the equation (9) continuously from 0to 1 over time, the black image may be controlled so as to graduallyreturn to the original avatar viewpoint image.

<Example of the Content where a Plurality of the Users Shares theVirtual World>

Although the above-described example is an example where one userexperiences the virtual world, there is a case in which the same virtualworld may be shared and experienced by a plurality of the users.

For example, as illustrated in FIG. 7, the scene is assumed that, on thevirtual world (presentation image) displayed on the HMD 11 of one user121A, the avatar 131B of the other user 121B is displayed, while, on thevirtual world (presentation image) displayed on the HMD 11 of the otheruser 121B, the avatar 131A of one user 121A is displayed.

The avatar 131A of the user 121A operates on the virtual world on thebasis of the sensor detection result of the user 121A. The avatar 131Bof the user 121B operates on the virtual world on the basis of thesensor detection result of the user 121B.

Therefore, the avatar 131 (131A and 131B) performs basically the samemotion as the operation of the user 121 (121A and 121B), but anoperation that does not match the operation of the user 121 may beperformed as the design of the content.

For example, a scene where the user 121A causes the avatar 131A of theuser 121A to bow a greeting is described below.

Although the actual user 121A may perform a bow operation, the avatar131A at which the user 121B looks in the virtual world may be caused toperform the bow operation since the user 121B may not see the actualappearance of the user 121A. Therefore, the user 121A performs anoperation instruction for causing the avatar 131A to bow by handling thecontroller 13, voice input, etc.

In such a scene, a situation described above occurs in which the userposture pos_(u) and the avatar posture pos_(a) are different, so thatthe presentation image generation process described in FIG. 6 may besimilarly applied.

That is, when the user 121A instructs the bowing operation with thehandling buttons, etc., the presentation image is displayed in which theavatar 131A of the user 121A performs the bowing operation on thedisplay unit 42 of the HMD 11 worn by the user 121B. When the avatar131A of the user 121A performs the bowing operation, the posturedifference between the avatar posture pos_(a) and the actual posturepos_(a) of the user 121A becomes equal to or greater than the firstrange. Then, as the process of step S6 in FIG. 6, the presentation imagegeneration unit 74 generates the reduction processing image and causesthe display unit 42 of the HMD 11 worn by the user 121A to display thereduction processing image. In other words, when the avatar viewpointimage according to the viewpoint of the avatar 131A performing the bowoperation is displayed on the display unit 42 of the HMD 11 worn by theuser 121A, the view change is large, and “sickness” occurs, so that theimage is switched to the reduction processing image.

Then, when the bowing operation is finished and the posture differencebetween the avatar posture pos_(a) and the actual posture pos_(u) of theuser 121A becomes within the second range, the avatar posture pos_(a) ofthe avatar 131A of the presentation image is controlled so as togradually approach the actual posture pos_(u) of the user 121A as theprocess of step 38 in FIG. 6.

In a case in which, what operation the user 121 causes the user's ownavatar 131 to perform is a selectable operation command, whether or notto present the reduction processing image for each operation command isdetermined in advance, and the presentation image prepared in advancemay be displayed in response to the instructed operation command. Thus,by displaying the image different from the viewpoint image of the avatarin response to the operation command in a case in which the posturedifference between the actual posture pos_(u) of the user based on thedetection result of detecting the motion of the user and the avatarposture pos_(a) occurs, the VR sickness may be suppressed.

<Exemplary Configuration of a Computer>

The above-described series of processes may be executed by hardware orsoftware. In a case in which the series of processes are executed bysoftware, a program that configures the software is installed on acomputer. Here, the computer includes, for example, a microcomputerincorporated in dedicated hardware, and a general-purpose personalcomputer capable of executing various functions by installing variousprograms.

FIG. 8 illustrates a block diagram representing an exemplaryconfiguration of a hardware configuration of a computer that executesthe above-described series of processes by a program.

In the computer, a CPU (Central Processing Unit) 201, a ROM (Read OnlyMemory) 202, and a RAM (Random Access Memory) 203 are mutually connectedby a bus 204.

Further, an input/output interface 205 is connected to the bus 204. Aninput unit 206, an output unit 207, a storage unit 208, a communicationunit 209, and a drive 210 are connected to the input/output interface205.

The input unit. 206 includes a keyboard, a mouse, a microphone, etc. Theoutput unit 207 includes a display, a speaker, etc. The storage unit 208includes a hard disk, a non-volatile memory, etc. The communication unit209 includes a network interface etc. The drive 210 drives a removablerecording medium 211 such as a magnetic disk, an optical disk, amagneto-optical disk, or a semiconductor memory.

In the computer configured as described above, for example, the CPU 201loads the program stored in the storage unit 208 into the RAM 203 viathe input/output interface 205 and the bus 204, and then executes theprogram, so that the series of processes described above are executed.

In the computer, the program may be installed in the storage unit 208via the input/output interface 205 by attaching the removable recordingmedium 211 to the drive 210. In addition, the program may be received bythe communication unit 209 via a wired or wireless transmission mediumsuch as a local area network, the Internet, or digital satellitebroadcasting, and then may be installed in the storage unit 208. Inaddition, the program may be installed in advance in the ROM 202 or thestorage unit 208.

In addition, the program executed by the computer may be a program thatperforms processing in chronological order according to the orderdescribed in this specification. The program may also be a program thatperforms processing in parallel, or at necessary timing such as when acall is made.

The steps described in the flowchart may be, of course, performed in thecase of chronological order according to the described order. Even ifthe processing is not performed necessarily in chronological order, thesteps may also be executed in parallel, or at necessary timing such aswhen a call is made.

In this specification, system means a set of a plurality of components(devices, modules (parts), etc.), and it does not matter whether all thecomponents are in the same housing or not. Therefore, a plurality ofdevices housed in separate housings and connected via a network, and onedevice housing a plurality of modules in one housing are all systems.

The embodiments of the present technique are not limited to theabove-described embodiments, and various modifications may be madewithout departing from the scope of the present technique.

Although the example of the head mounted display (HMD) has beendemonstrated as a display device which displays the presentation imagepresented to the user in the embodiment mentioned above, the head updisplay (HUD) or a dome-shaped (hemispherical) display etc. may besufficient as the display device of the imaging system 1. A displaydevice that displays a video so as to cover the field of view of theuser may also be sufficient.

For example, a form in which some of the above-described embodiments arecombined as appropriate may be employed.

The present technique may have a cloud computing configuration in whichone function is shared and processed in cooperation with one another bya plurality of devices via a network.

Further, each step described in the above-described flowchart may beexecuted by one device or may be executed in a shared manner by aplurality of devices.

Furthermore, in a case in which a plurality of processes are included inone step, the plurality of processes included in the one step may beexecuted in a shared manner by a plurality of devices in addition tobeing executed by one device.

The effects described in the present specification are merely examplesand are not limited, and effects other than those described in thepresent specification may be obtained.

In addition, the present technique may also have the followingconfigurations.

(1) An image processing device including:

an image generation unit configured to generate an avatar viewpointimage according to a viewpoint of an avatar corresponding to a user in avirtual world as a first presentation image to be presented to the user,and generate a second presentation image different from the avatarviewpoint image in a case in which a posture difference occurs betweenan actual posture of the user based on a result of detecting motion ofthe user and a posture of the avatar.

(2) The image processing device according to (1), in which

the posture is defined by a head position and a gaze direction, and

the image generation unit generates the second presentation imagedifferent from the avatar viewpoint image in a case in which adifference in azimuth angle or elevation angle between the actualposture of the user and the posture of the avatar is equal to or greaterthan a first threshold.

(3) The image processing device according to (1), in which

the posture is defined by a head position and a gaze direction, and

the image generation unit generates the second presentation imagedifferent from the avatar viewpoint image in a case in which adifference in azimuth angle or elevation angle between the actualposture of the user and the posture of the avatar is equal to or greaterthan a first threshold, or in a case in which a difference in a headposition between the actual posture of the user and the posture of theavatar is equal to or greater than a first threshold.

(4) The image processing device according to any one of the above (1) to(3), in which

the second presentation image includes an image corresponding to aposture of the user.

(5) The image processing device according to any one of the above (1) to(3), in which

the second presentation image includes an image corresponding to a headposition of the posture of the avatar and a gaze direction of a postureof the user.

(6) The image processing device according to any one of the above (1) to(3), in which

the second presentation image includes an image corresponding to a headposition and an azimuth angle of the posture of the avatar, and anelevation angle of a posture of the user.

(7) The image processing device according to any one of the above (1) to(3), in which

the second presentation image includes an image corresponding to a headposition and an elevation angle of the posture of the avatar, and anazimuth angle of a posture of the user.

(8) The image processing device according to any one of the above (1) to(7), in which

the image generation unit generates the second presentation imagedifferent from the avatar viewpoint image when a case in which theposture difference between the actual posture of the user and theposture of the avatar is equal to or greater than a first thresholdoccurs for a predetermined number of times or continuously for apredetermined period of time.

(9) The image processing device according to any one of the above (1) to(8), in which

the image generation unit further generates the second presentationimage so as to gradually approach the avatar viewpoint image in a casein which the posture difference between the actual posture of the userand the posture of the avatar is within a second threshold.

(10) The image processing device according to any one of the above (1)to (9), further including:

a handling unit configured to instruct an operation of the avatarcorresponding to the user,

in which the image generation unit generates the second presentationimage different from the avatar viewpoint image in a case in which theposture difference occurs between the actual posture of the user and theposture of the avatar by operating the avatar in response to aninstruction from the handling unit.

(11) An image processing method including the step of:

by an image processing device,

generating an avatar viewpoint image according to a viewpoint of anavatar corresponding to a user in a virtual world as a firstpresentation image to be presented to the user and generating a secondpresentation image different from the avatar viewpoint image in a casein which a posture difference occurs between an actual posture of theuser based on a result of detecting motion of the user and a posture ofthe avatar.

(12) An image system including:

an image generation unit configured to generate an avatar viewpointimage according to a viewpoint of an avatar corresponding to a user in avirtual world as a first presentation image to be presented to the user,and generate a second presentation image different from the avatarviewpoint image in a case in which a posture difference occurs betweenan actual posture of the user based on a result of detecting motion ofthe user and a posture of the avatar; and

a display unit configured to display the first presentation image andthe second presentation image.

REFERENCE SIGNS LIST

-   -   1 Image system, 11 HMD, 12 image processing device, 13        Controller, 41 Sensor, 42 Display unit, 62 Image generation        unit, 71 Avatar operation control unit, 72 User posture        detection unit, 73 Posture difference calculation unit, 74        Presentation image generation unit, 81 Handling unit, 201 CPU,        202 ROM, 203 RAM, 206 Input unit, 207 Output unit, 208 Storage        unit, 209 Communication unit, 210 Drive

The invention claimed is:
 1. An image processing device comprising: animage generation unit configured to generate an avatar viewpoint imageaccording to a viewpoint of an avatar corresponding to a user in avirtual world as a first presentation image to be presented to the user,and generate a second presentation image different from the avatarviewpoint image in a case in which a posture difference occurs betweenan actual posture of the user based on a result of detecting motion ofthe user and a posture of the avatar, wherein the actual posture and theposture of the avatar are each defined by a plurality of parameters,wherein the first presentation image includes an image corresponding toeach parameter of the plurality of parameters defining the posture ofthe avatar and the second presentation image includes an image notcorresponding to each parameter of the plurality of parameters definingthe posture of the avatar, and wherein the image generation unit isimplemented via at least one processor.
 2. The image processing deviceaccording to claim 1, wherein the plurality of parameters include a headposition and a gaze direction, and the image generation unit generatesthe second presentation image different from the avatar viewpoint imagein a case in which a difference in azimuth angle or elevation anglebetween the actual posture of the user and the posture of the avatar isequal to or greater than a first threshold.
 3. The image processingdevice according to claim 1, wherein the plurality of parameters includea head position and a gaze direction, and the image generation unitgenerates the second presentation image different from the avatarviewpoint image in a case in which a difference in azimuth angle orelevation angle between the actual posture of the user and the postureof the avatar is equal to or greater than a first threshold, or in acase in which a difference in a head position between the actual postureof the user and the posture of the avatar is equal to or greater than afirst threshold.
 4. The image processing device according to claim 1,wherein the second presentation image includes an image corresponding toa posture of the user.
 5. The image processing device according to claim1, wherein the second presentation image includes an image correspondingto a head position of the posture of the avatar and a gaze direction ofa posture of the user.
 6. The image processing device according to claim1, wherein the second presentation image includes an image correspondingto a head position and an azimuth angle of the posture of the avatar,and an elevation angle of a posture of the user.
 7. The image processingdevice according to claim 1, wherein the second presentation imageincludes an image corresponding to a head position and an elevationangle of the posture of the avatar, and an azimuth angle of a posture ofthe user.
 8. The image processing device according to claim 1, whereinthe image generation unit generates the second presentation imagedifferent from the avatar viewpoint image when a case in which theposture difference between the actual posture of the user and theposture of the avatar is equal to or greater than a first thresholdoccurs for a predetermined number of times or continuously for apredetermined period of time.
 9. The image processing device accordingto claim 1, wherein the image generation unit further generates thesecond presentation image so as to gradually approach the avatarviewpoint image in a case in which the posture difference between theactual posture of the user and the posture of the avatar is within asecond threshold.
 10. The image processing device according to claim 1,further comprising: a handling unit configured to instruct an operationof the avatar corresponding to the user, wherein the image generationunit generates the second presentation image different from the avatarviewpoint image in a case in which the posture difference occurs betweenthe actual posture of the user and the posture of the avatar byoperating the avatar in response to an instruction from the handlingunit.
 11. An image processing method comprising: generating an avatarviewpoint image according to a viewpoint of an avatar corresponding to auser in a virtual world as a first presentation image to be presented tothe user and generating a second presentation image different from theavatar viewpoint image in a case in which a posture difference occursbetween an actual posture of the user based on a result of detectingmotion of the user and a posture of the avatar; wherein the actualposture and the posture of the avatar are each defined by a plurality ofparameters, and wherein the first presentation image includes an imagecorresponding to each parameter of the plurality of parameters definingthe posture of the avatar and the second presentation image includes animage not corresponding to each parameter of the plurality of parametersdefining the posture of the avatar.
 12. An image system comprising: animage generation unit configured to generate an avatar viewpoint imageaccording to a viewpoint of an avatar corresponding to a user in avirtual world as a first presentation image to be presented to the user,and generate a second presentation image different from the avatarviewpoint image in a case in which a posture difference occurs betweenan actual posture of the user based on a result of detecting motion ofthe user and a posture of the avatar; and a display configured todisplay the first presentation image and the second presentation image,wherein the actual posture and the posture of the avatar are eachdefined by a plurality of parameters, wherein the first presentationimage includes an image corresponding to each parameter of the pluralityof parameters defining the posture of the avatar and the secondpresentation image includes an image not corresponding to each parameterof the plurality of parameters defining the posture of the avatar, andwherein the image generation unit is implemented via at least oneprocessor.
 13. A non-transitory computer-readable medium having embodiedthereon a program, which when executed by a computer causes the computerto execute an image processing method, the method comprising: generatingan avatar viewpoint image according to a viewpoint of an avatarcorresponding to a user in a virtual world as a first presentation imageto be presented to the user and generating a second presentation imagedifferent from the avatar viewpoint image in a case in which a posturedifference occurs between an actual posture of the user based on aresult of detecting motion of the user and a posture of the avatar,wherein the actual posture and the posture of the avatar are eachdefined by a plurality of parameters, and wherein the first presentationimage includes an image corresponding to each parameter of the pluralityof parameters defining the posture of the avatar and the secondpresentation image includes an image not corresponding to each parameterof the plurality of parameters defining the posture of the avatar.